feat : Add orphan checkpoint retention policy #28
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
@Parthiba-Hazra Would it be possible to rebase this pull request on the main branch? |
- Introduced global, container, pod, and namespace-level policies for checkpoint retention, based on storage/size limits. - Updated CRD definitions to store the storage/size based policies. - Updated the sample configuration of CheckpointRestoreOperator with storage/checkpoint-size based policies Signed-off-by: Parthiba-Hazra <[email protected]>
- Enhance generate_checkpoint_tar.sh to optionally generate tar files larger than 5MB - Update GitHub Actions workflow to test storage quota garbage collection policies Signed-off-by: Parthiba-Hazra <[email protected]>
Signed-off-by: Parthiba-Hazra <[email protected]>
- To implement the orphan checkpoint retention policy, the manager requires permissions to watch and get resources. This allows the manager pod to watch the relevant resources and retrieve the necessary resource information when applying the policies. Signed-off-by: Parthiba-Hazra <[email protected]>
- Added support for orphan retention policies at the global, namespace, pod, and container levels. - Introduced the `retainOrphan` field in each policy type to control the retention of orphan checkpoints. - Updated the policy application logic to delete all orphan checkpoints when `retainOrphan` is set to false. - Implemented a PodWatcher to monitor pod deletions and apply policies immediately when a resource is deleted. Signed-off-by: Parthiba-Hazra <[email protected]>
- Add `test_orphan_retention_policy` test and update GitHub Actions workflow to test orphan retention policy Signed-off-by: Parthiba-Hazra <[email protected]>
Signed-off-by: Parthiba-Hazra <[email protected]>
rst0git
reviewed
Oct 30, 2024
| - `checkpointDirectory`: Specifies the directory where checkpoints are stored. | ||
| - `applyPoliciesImmediately`: If set to `true`, the policies are applied immediately. If `false` (default value), they are applied after new checkpoint creation. | ||
| - `globalPolicy`: Defines global checkpoint retention limits. | ||
| - `retainOrphan`: If set to `true` (default), orphan checkpoints (checkpoints whose associated resources have been deleted) will be retained. If set to `false`, orphan checkpoints will be automatically deleted. |
There was a problem hiding this comment.
Suggested change
| - `retainOrphan`: If set to `true` (default), orphan checkpoints (checkpoints whose associated resources have been deleted) will be retained. If set to `false`, orphan checkpoints will be automatically deleted. | |
| - `retainOrphan`: If set to `true` (default), orphan checkpoints (checkpoints whose associated resources have been deleted) will be retained. If set to `false`, orphan checkpoints will be automatically deleted. This is particularly useful for transient checkpoints used to recover from errors by replacing 'container restart' with 'container restore'. |
|
Closing in favour of #60 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR introduces the orphan checkpoint retention policy, allowing users to control whether orphaned checkpoints are retained or deleted.
retainOrphanfield is added across global, container, pod, and namespace policies.retainOrphanfield tofalse.